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. ' ' " AiDstr^t 

This paper i-s concernexl witjff th^- process of ^hiiman reading as • 
, a high-level p^erceptual ^task. Drawing on insights from 
Artificial Intelligence research ^ — s-peciflcally ^ research in 
natural language processing aad continuous speech uplerstandin^g 
— 'the paper attempts to present a fairlyconcr^te picture of the 
kinds of hypothe§is formation and inference^ processes that must 
take plae^v during^ the reading process. It Makes the .case that 
many 'more alternative hypotheses are formed and evaluated than 
one might expect and that the majority of such processing happens 
tyelaw the level of 'conscious introspection^ Two text" passages of 
different, levels "'of .difficulty are considered in det^ail, gnd the 
applfcability of such Artificial " Intelligence insights- to the 
modeiing of human reading are discussed'.. - ■' !L - - ' 
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/ . . ^ Multiple 'Theory Formation'. 

\1 Overview ^1^. 

-'')-■-■ ■ : ■ ■■ ' " - ... 

High-level perceptual tasks' such as reading, spe'ech 
understanding, 'and visual scene interpretation are characterized^ 
by the need to discoveria Structured interpret^at ion that accounts 
for th*e stimuli pre^en^.^ This process is prerequisite to 
' deciding what has been perceived, and thus precedes whatever 
process decides what to dp with the resulting perception — what 
significance' to* attach to it, whether' to remember it, how to 
incorporate it into the knowledge base of the perceiver, etc. In 
this paper , ' I .will attempt to make the "'case that the process of 
^ arriving at an interpretation of th$ input involves the foi?mation 
and ^evaluation of many alternative , part ial hypotheses about the 
input and that this process gdes on largely below the^^^^Te^^el of 4 
. introspective- awareness^ of 't^he "perceiver. Even" though skilled 
reading involves a variety of "metaoognit ive" strategies [Brown, 
J978]j , in normal reading these processes are themselves invoked 
without conscious attention to the process of doing so." 

•1 will focus on the problem' of ^reading , and .jwill' draw on 
. . iQsights a(id analo^gies from work in, Natural -Language - Parsing and 
'Continuous §peech ynclerstandiog . Since 1 d6 not liave ^space here- 
to giv^ an adequate dntroductioji • to ^ all of. the "bacjcgpound - 
' material that I would like to use^ I .\will -instead refer "^Jj^e 
reader to * three previou^ papers: "Meartihg and Machines" [Woods,' 
1973b], "Syntax, Semantics/ and. Speech" [Wooys, 1975a] ^ and . 
-"What's, in Link: Foundations \or Semantic Netv^orks" [Woods, 

ERIC • • . . , , ^ • . , 



• ' < •• - . .Multiple Ttjeopy Formation* 

197-5b], /the most recent - material on speecti* 'understanding* " is 
.-unfortufiat^ely only contained in' technical reports.' I recommend 

report [Woods et al . , . t976"^ , Vols, I, 
I-II,-IV, and v.. For a brief overview of the^ BBN speech 
understanding system, seeWolf-and Woods [1977]. 

2.. Multiple > Hypoth'ese^s, Backtracking, and' .Nondetermi ni sm 

- o""^ iJituitions about techniques for dealing with 

multiple hypotheses pome from work . tin " par-sing algorithms foe 
formal grammars.- This, is an area whe'he both theoretical results 
in-^autdmata theory and empirical results from programmed' parsers 
are available. Other^. areas 'where such in-sights can be" gained" .are 

-formal' theorem proving, machine < vision , and continuous speech 
understanding.^ Perhaps , the earliest and most widely known 
mechanism for ' handlinig multiple .hypotheses is the predicative' 
analyzer, or pushdown store parser "[Ku;io &' Oettinger,' '1963], 
'••using backtracking '.to handle the possibility , of multiple 

' alternatives. ' ■ , ' " ^ • 



s 



- ^ A pusljidow-n-^stcire parser is essentially • an' algorithm • for ^ 

in terms ' oT rules ■■ such as "a sentence 

consists af a . noun' phrase .followed by a * verb .phrase" " it 

operates oy, using a stack or "pushdown, store" of words 

' •■• . — — i ^ 

phrases that- are expected -to ocgur -iii the -sentence ' and 'at each 

' » ' . . f 

step- compares 'the>next in^jut ,word against the top item on the 

stack, either .finding a match between the current word and the' 
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predicted ,category,., finding tha't the current word iS'-inr^^- 



With the predicted categpry, tor expanding .a predicted phra^l ^. 
to a sequence of words ajid phrases„ that would realize^ tt \C*3h^^^ 
• which case the new predicted categories* replace the; one ft;^ 
which they, are derived on the stack): The stack operates *ve^v\^\^^v|^^ 
much like the spring loaded stacks of dishes and trays that onsi^^ 
f-irmts'^ in some cafeterias' (from wfiich it derives its name)'%f^^S| 
Backtracking^ refers to the process of 'saving enough information 
t>efore making a choice among alternatives (in t'his^case before 
* choosing a particular rule to expand a phrase type) <bo that at a 
later titne the situation, {ffior ^to the choi-ae can be recojnstructed 
.and a-dif fe,rent choice selected. . ^ 

• For ^ example , if. a backtracking parsed encounters a wopd that 
can bgieither a noun or a verb ^in a context where both nouns and 
verbs woulxl bjf- acceptable (albeit wi^th different interpretations) ' • ^ 
^ theo enpu^ information is ^aved to* remember the current position 
'.in the sen-tence "and all of tine' decisions made so far. before 

' • , . . ^ ^ , ; ' 

making on^ of 4:he choices, say tr^aXing'^ the word ^as a noun. 
Subseq'uently if this ' choice doe^s not lead- -to 'an acceptable . 
complete parsing' (or ih' any case if all possible parsings are* 
Ceguirecf) , the saved inftfrmation is restored. and the other. choice 
vis tried. I ] 

> ' ■/ . - ... " ' ' 

Backtracking is not' the only -way . 'bf- ' impiementlng an i .) 

./ • . ' - V ■ ^ 

algorifhra for expl.ot'irig.' ' niultiple alt'ernatives, 'however , Other 

^methods involve the creation oif separat-fe virtual .processes 'foh' >^ 

: ' ■ ■ . ' s ' 
' - • ' ■ ' •. ■ ' . . 
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^each .of the,^ alternative choloes ; which c^n-then be ruji in 
.parallel or in ^ some prioritized order. U virtual process 
behaves conceptually as if it wer'e an ongoing 'procegs , but may in 
fact K be*, competing for time a'nd resourced " with other^ such 
processes so that the -'number, of. processes - actually receiving 
.resources at any given moment may be fewer than t^e- number of 
"virtual" processes th.aJt are conceptually "active".) 

. , ^A^eral method for specifying and discassing algorithms 
• for this kind of processing of multiple alternatives is to Er-eat 
'tbe .algo/ritJiras as procedures one of who'se , basic i^nstructiqn type^ 
has-fehe meaning "choose one of the following alternatives-: 
Such pr})cedures 'are "executed"" 'b^ an interpreH^er that 
'systematically considers all ppssible comb/nations "..of such- 
choices, e/ther -by backtracking or by \he method of separate 
virtual processes . Such' algorithms are referred ' to -as" 
non^eterministic algprithms, not because their evenj^ual behavior 
is not determined,, but because the spebif ication .of. the exact 
drde'r in which - the various alternatives will be consid^d^and " 
the .mechanism by" which the dombinations of . choices will be 
enumerated are not- overtly specified as part 'of the algorithnj'; ' 

Parsing natural language is fundamentally a nondeterminisUc 
process.-. That is, it is -not- possible fe-o- make-all ' of-the uu r-re' o -t- 
,glecisions faa%ed solely on local information./ -"father tentative 
decisions mus^t be made explored to see^ their consequences 
are plausib'u. For' example, \he decision as t6' whether .a given'"^^ 
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word . i 6 being used as a noun or a verb depends on whether a' 
complete pars'iog' can .be. found under one or the other of the two 
assumptions. Thus "Fly" in "Fly fishing is furl" 'is a noun (used* 
as*^ a modifier), while^in^ "Fly fishing equipment t'o^ Lakeview Lodge 
.as soon as possible" it is- a verb. • (If you required a few 
seconds to find the Corrfect interpretation of this sentence jlue 
to an initial inporrect interpretation .of "Fly fishing ," ,then you 
have experienced what is known as^.the "garden path" phenomenon.) 
Occasionally, several choices will lead to a domplete parsing, in 

* ' r * 

which case the sentence ' is ambiguous (as in "The girl guides* 

' fish" , ^which could either be- a statement about .what girl guides 

do in- the.ir spare time or a* descriptiqn/of a shepherdess for 
'fish). ' 

Predictive' parsing is an intuitively satisfying analogy for 
thinking 'ab^ut hun^an sentence processing sin|e it seems to 
c6rrespond weil to our introspective awareness of- stages of the 
parsing .process. Notably, if, the sentence ^is stopped at some 
p*oint w% can generate^ a 'completion ; in general., we know what 
kinds of words to expec*^ next » at • each' ppint; and occasionally we 
are aware 'of having ms^de a false early decision and having feo^ 

t • 

back up to look, for another alternative (in gai?den path sentences, 
like the one ^above). One soon gets into difficulty with this 
analogy, however, sinxje 'systematic predictive analysis for a 
hat-ural^ language' grammar makes many mor^ extrdffeous tentative 
excursions down false parsing patha than people ^^m to. Also, 
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the tiflle required- by- people to/r*c6\rer frpm a, garden ^ path seems 

to be. much s.hort^ than that re^uirecj' by a ' backtracking^'^parser 

[Collins & Quillian, 1971]/ In 't-he^ Artificial inteliigenqd 

field, this has' given f^ise t/ ^. search -for techni(qu^s^ to make the 

enumeration^of parse patMs more selective' (e;g,-, - by using 

"semantic" informat.ion )^ ]Ln order to correspond more close^^y to 

the level of backtracking/ that we are aware of when we read. In' 

^ ~ -I ^ " 

^other x^uarters, it hai given' >r,ise ^'to tfi% labeling of such. 

» / . * 

.processes as "Gestalt".,/ treating the progress a black box whos'e^ 

internal workings one /doea not attempt, to understand, 

/ * 

- In th|s .paper I will attempt to make the' case for another 
account, namely that the amount of partial 'hypothesis formation 
that a person actually performs in such' task? is far more than it 
^seems, and that .the bulk of it' occurs . beldw, the- level of 
in.troapective kwarene§*s. ^ I will draw largely ^on ^experience* in 
constructing speech understanding systems\ • In the ^speech 
understanding taTsk, it is relatively .easy to make thev case that 
'ther.e ^s a great deal of hypothesis ' formation and testing 
required 5ven to'know wh^t words have been heard, much less how/ 
they 'are to be parsed and interpreted. Hov^er, most of this 
processing tak-es placte exceedingly fast' and witKout conscious 
effort. , Once it is realized that the human perJceptual processes * 
are capa'ble of making large numbers of alternative hypotheses and 
choosing among thfem ^on . the basis ' of relatively h-igh-level 
plausibility judgments - all without, conscious effort or 
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vawareness- it is not difficult" to imagine such oapabilities- 

' ' ' ^ ^ \ 

'Dei'ng used in r*eading and other high-level perceptual tasks,- 

» ' ** * 

"3. Cnaracteristios of the Speech Understanding Process , 

;A naive view of speech understanding' might consider. it as ^a' 

•process of successively recognizing speech ..sounds ''(called' 

phoneme^), grouping^ phonemes into words, parsing word seq^uences 

into sentences, and finally interpreting the. meanings of*' those 

sentences. However, considerable experience now . indicates that 

• the acoustic evidence present in the original - .speech signal is. 

,not sufficient to support such a process [Woods and Makhoul , 

1974]. For sentence^ 'recorded ^om continuous speech, it is not 

generally possible to reliably determine Jthe phonetic identity of 

the individual 't5^oj:ieines ^ or even to be sdre^ ho\^ many phonemes are 

present) using the acouskis-^vidence alon^. Experiments in 
. " * ^ . , , ' i • 

spectrogram reading [Klatt and Stevens, 1971] indicate that the 

*■* - ♦ 

reliability of such determinations can bfe increased by use of, the 
redundancy ^pr*oVided by 4<nowledge of the vocabulary', the^.^syntax of 
the language. , and semantic and- pragmatic consi<Jera tions 

^ Tape splicing experiments [Wanner, 1973] seem to indicate 
fehat this low-le-vel acoustic/^ambiguity is ^ an inherent 
characteristic "^'of speech ailcl nqt just' a limitation of ^hUman 
spectrogram-reading • SpecificalJ^ , intelligibility of individual 
worrds excised from ccyitinuous speech is very low, but the 
intelligibility* increases wheh sequences of two or three word^^ 
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are" excised. It appears that'fhe ajiditional .copsfraiht> of having , 

to make seftse in a. larger context ^^^gins to rAs^dlve .the 

, ' ^ — ^ . ; , ^ ^-^^ ' " ^-^"^^^ ^ ^ ' ■ ' 

* ambiguities t)iat «were pre^fent whep-^ly ^ihe ^c6ustic evidenqe was 

considered. This proce^sixigrThow^ver, • happens below the level of 

iritrbspection, an^t has air^o-f the ^sufejedtive ch^racteristijcs of a' 

wholistic or GesJ^jilt phejipmenon . - Th>t i$, ,if a sufficiently long 

s^quenc^ie of continuous ^ speech i^ ' * heard , its. correct 

W ^ ' t 

^interpretation , usually appea^r-s immediately and 'effortlessly, 

. with^out ^ conscious Rareness of the' defails of the^^^gjvoegSs . The 

vast majori|fcV;of oyr s^Roken comm^unicatio^ in thii. ^ 

manner, and it is markedly .contcasted. with '^ose cases • wher*e . an ^ 

utte/*ance ^ is garbled sufficiently to invoke conscious effort to 

decide, -whaf was said. — ^ . . ' ^ - 



* TKiebries. Motiitora- ^ 'l^otices . and Ey^nts" 

• A( Compufflitiortal- Fi'^amework for Perce^ion " 

SThe BBN^sf^eoh un-derstandrng "System [Woods et ^1,, 1976-; 
^ Wolf ,and Woods, ^ 1977] has -evolved within a general framewofk fdr 
/ viewing perceptual processes'. Central to'th-i^ framework is an' 
entity ^call^d a theory . A theory represents a partioular 
hypothesis about s6me or all of 'the sensory. stimXali that ' are 
pr^esent* Perception is ^vievied as the process of forming a 
believable"coherent theory which ^an account for all the stimuli/ 
*This is arrived ^t by_ successive'" refinement and ejctension of 
partial theories until^a best complete theory is found. 
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i, a hi^lj^lfevel jp^'cepti|c^h\4>r6eess requires the 

tialty -infinite , class 



* ability 'io .reobgnize any menii^^lTof; a ^'^^^^ 

\oS _ per'c^ptil^e ot?M^«jrs''\ that ?rV"^bartsi^5uVt;^f onjt of.^eaementary 
constituent^>e^^r-ding, to k-ripwn ^rul^es.' ' Thaf is,, t'he- object 
'is -gene/al^ly a. compo^d object, constFjicted rrom 



perc< 



members (4;^ a finite se,t ^of el^fentary constituents ^ a^cording-s to 



rules. ■ .These " elementary 



cind ^-of welll-fomedrtess, 
co1is/ituej>trs, /as; well/as/the. relationships' among Jth^ ' that are 

mvoKed m the /well-formedness rules,- musf'^e directive 

.^^ • ' ' • « • . • . • * 

perp.eptible. Tjius, a perceptual' system 'must ^ incorporate somS 

basic epis^t^o logical, assumptions about the kinds of tiling? that' 
it can^xi5^eive and'th.e' .rules , g'oTerning their assertfbly. ^ The. 
w^-formedness \- rules can- be "used to :'' re jec-t impossible. 
. interpretations yof the input stimixli, an-d may als^o be useab.le..to 
predict- Other,, constituents that, could be preisent_„iX„.a given.' 
.partial* theory is cornet; 



. Thi? -perception framewoi^k ^ssuraes mechan'isnjs " Vof .using 
subsets • a.f . the input stimuli to forpi initial "seed" Hypptheses 
for certain ■ elementary cotist^ituents - (stijrful,u-$-driven 
hypothesization) and ' me?Dhanisms for ■ deriving hypotheses for* 
addit^onai compatible constituenifs'' from a. partial', '^t-heory- 
(theory-driven^ or predicted, hypothesization* ) It also assumes ' 
^mechanisms for' verifying a hypothes is^against/the input "stimuli ' 
, and evaluating! the well-^ormedngss of. a .compciund 'hypothesis to. 
AssigTiJCit Cori| measure o5 quality and/or likelihood.' A t-heory ' 



•4 
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may tl)erefore be tliought /of ^as a ; hypothesis that has been 
evaluated in this way and assigned a measure of con^idenfee " 



" Irf. the case^of speech understanding, a theory 'can range from 
an ^lem^tTary hypoEThesis t^iat a particular word is^^present , at a 
par.tidtolar' ' p^int* In' , ifhe input . (a word matt&h ) to a. complete 
hypothesis, gf 'a' covep^ng-^ se(iuence ^Cf words with , a complete . 
s'yntacltic and semantic interpretation, (In ^erjeral a thiory can/ 
' be se.t of compatible word :hypo.the§es with gaps between the.m and^ 
with p^artial syntactic and semantic interpretations ]^\^A partial.^ 
theory .rpay be able to generate predictions for approorlale ^ words , 
o| classes, of words e-^ther Adjacent to ^JW},^ ' wor^ds^ alread/' 
hypothesized, or {)bssibty elsewhere in the utterance. - ) ^ 

• ' . ■ ^ . / \ . " ^-^^ \ tv '. 

Predictions . are dealt, with in our computational fRamework by i 
two kinds of* devices: monrtqrs / which are passively waiting foV' ^ 
expected constituents? and proposals ^ .whic>i are elementary 
hypotheses that are *to be evaluated against. the input. ^ PrjDposals. 
result in ,ac,tively seeking stimuli that would. verify them, while ^ 
monitors passively wait for such hjrpotriesQs .to be formed. 'Thfe 
functioning of moa^^t/prs assumes that • there is^ an orga^rizing 



f^Our notion o*f •'Stimulus-driven hypothesization is essentially the 
.same as .that of '"bofat6m'-up" * processing r^fe^^d " to in w many ^ 
discussions ' of , sUch ^ ^rpaesses . ^ , .Howevei^ our "notion of 
^theory-driven, hypobhesizat lan , is si igt^ly ^ dif tbr^ent 'from the 
^^ens^ ^usually given to "tOD-dpwn-" processing ,in that it doea not 
"necessarily imply- any glob'al^ '("topmost"^) 'hypothesis, but .only 
predictability >by' som6 .other .hypothes\s^ which may itself have t 
been derived "bottotn-up."'. The. term's ^^top-down"^ and ^ . "'bottfom-up'! " 
in this • sense' * come from ^ the literature . on «f6nrn^i Da4|Qjing 
algorithms. ' ' . ^ ^ ^^^i. ^ ■ 



MultipiS Theory Formation 

'structure into which all derived partial jiypotheses are placea .aa 
^they ar^ discovered and that/the^ monitorvS can essentially set/, 
• !'trkpsi5|\in this str^ct.ure for, the kinds of events that -4:hey. are 
watching for^Thi^s is tol iJe cohtrasted with continuous "^parallel 
evaluation, o'f special pro'cea^ses (frequently called '[deroo'ns")' to^ 
watch for expected patterns in the input ^stream. Monitors. • 

perform no 'computatipn unti/l.and unless some" other process makes 

. ' . / ' . ^ ' ( 

an entry ^of the kind 'they^are- waiting for in- some data structur 




The 'functioning of, monitors^ * is "Illustrated by an early' 
• s^peeoh understandi^ng system .dealing with concentrations of 
"""chemical- . Blemen^S nil lunar rocks; ' There', Tor example , "a' word 
^mat^h foHjJll^'onc Strati on" weoild S^et .monitors 'on- the concept nodes 
fo'r SAMPLE and CHEMICAL ELEMENT in the semar^tic network* . If a 



word such as "Heliikm"^ was subsequently found anywhere els6 in the 

' . . , . 1 * ^ ^ 

utterance, a checlc^a, the semantic network starting with HeliunK^ 

wbuld lead to the superset dategory CHE>J^ftL ' EL^ENf where it 

would w^ake up the monitor from "eoncentf'at ion" , thus detecting 

the coinc^idence af a^ detected .hypothesis * and a pr^edicted 

hypothesis [Nash-Webb^r , '1973^'. ^ ' * . ^ 

When a monitor is triggered, an event is created 'calling for 

'the evaluation "of a new hypothesis and the creation of ' a new 
..... . . - : • 

theory if theH^ypothesis is not rejected. In ga.neral , a number 

of events are . competing for service by the-^ prp^esSor at any 

moment. In .human perception,^ there jjj^ay be full parallel* 

processing of su^jh .events, but in a serial mac'hine, these events 
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must be^ queuecS^nd given processing resource3 , on the basis oT 
some priority ordering. (Even in human . perception , .there is 
probajDly some sort -of priority all</cation of resources, since 
yarious kinds of inte^erence can ecour.> In our * computational 
fram^^ork, events ar^ maintained on a queue in order of priority, 
the top event being processed|^t each step. • ' , 

The processing of an everrt can result- in new proposals being 
made, new rnonitors being set, and existing monitors being 

J^liggered to produce .new* events. Since so much hinges on the 
event chosen for , processing , a major issusi is tha»t of -assigning- 

"priorities " to event? ~ Oifd^r ' to find - .the^^inost '^likely 

Interpretation of the input. In the BBN* system, priority^ scores 
are assigned on the basis o/r ^ayesian estimates of the 
probabilities of the competing theories,- and certain Gontr*ol/' 



strategies and priority scoring me^jylcs can be guaranteed to 
discover 'the most probaiie interpolation of the input.* 



5. ^ 'Control Strategi-es - ^ ' ■ v . 

The above discussion leaves open issues sUch as iwhen should, 
seeds be formed, how many sh_ould be considered, should'^11 seeds 
be'worJced on in parallel , et*c . * Thes£e ^issues' we refer to as 
control * •issues. . They h^ve been critically ' important \n 
computerized speech' understanding systems *-*-In ^ the BBN system, 
for example, there are a variety of differ^ t control strategies 
that all fit within the above , paradigm . Figure 1 illustrates-one 
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class of strategies in which seeds are formed anywhere in the^ 
""Utterance* that sufficiently salient word matches are founds The 
figure shows the seed events formed as a* result of •an initial 
scan of an .utterance for high lik'elih<5od word matches -anywhere in 
the utterance. ..EacH^ theory is assigned a score expressing ,its^ 
likelihood of being correct (actually a logarithm- of the^ rat^io of 
the likelihood of th'e-^ acoustic evidence given the theory over th( 
a priori likelihood of that evidence occurring independently) 
The region of the utterance covered by tfie theocyr is iiTdicat/d by 
specifying its 'left 'and ' right , boundary positions in a list of 
potential bXindary positions (the Jleft end of the utterance i3 
numbered .0^ and in.*-this ca^^ -the right end is. numbered l8). T4ie 
.yxclam^ation marka^ndicate the^ theories " that are actually part of 
the correct interpretation. ' / 

For this general class .of control strategies, referred to as 
*^middle-out" , theories are 'grown by starting- with a seed word, 
asking; a higher-level .linguist ic component to predict categories 
(?f words that" can occur on either side .of ''it, asking a lexical 
retrieval com^^ent to f,ind-* the matching words in tho^e 

categories on the appropriate sides, and generating events for 
Qach such word ;^dijml to extend the theory b^ [adding' that word. 
Thus, events will be, placed on the event x^ueue to add^wordjs both 

r 

on. the left and on the right ends of given theories . These- "new 

:^ • " ^ 

word" events will compete with each other and with the remaining- 

seed events on the ba3-is of sjore to determine which event will* 



. \ , . ' ■ ■ • 
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be f^Hocessed next, causing'-th? processor to sometimes .continue 
adding wor4s- to .a given 'theory and at other timejr to shift its 
processing to a different competing 'theory . , ** 

. figure 2 shows the sequence' of theories that are fo'rm^d as a 
result o|* t^j,s. process , sjtarting with the event queue of ^ Figure 
^ (Buckets in the figure indicate theories that include the^ 

' hypothesis thgt the left or right ends *o/ the utteranc^have been 
reached. A number ir> -parentheses after a theory number is' the 
number of a;preceding theory from whifrh tjie indicated theory- was, 
forme d;by the addition of»a new wandV) • Notice^ that the final 
theory . is , developed in -this case by working Undependjently on two 
difr^rent p(jirtions ^of the utte^^ance starting from the .^eeds. 
"Show" an:d "trips'".,/ The 'fi^al .theory in figure d is in fact 
dejiv-ed from a kind of evewt called a collision ^ytnt wh^^c'h' 
•combines "the . theories "shov'nie^^''Tnd~^' trips"- when they both notice 
the word '"her" filling the ga/p between them. This event is' 
formed during the processing of theory 13, although its score is 
:such that it does nqt' reach the top of . the queue until theory 23. 

Figure' 3 shows the seed theor'les for a hybrj^'d strategy- in 
which seeds are started withiij^a bounded distance from the left 
end of the "utterance,^ and are* grown right-to-left' until .they 
^t^achr the left errd^' af#er which' the^ remainder of the processing 

is left-to-rightT . jFigure 4 shows the sequence of theories' 

' > i ^ ' . , \ ' 

developed ini the* course of ^ un'derstanding this utterance using the 

' ' r ' . ' ' * 

hybrid strategy , The ^basically ^ left-to-right nature of the 

r r.. ' • - is : ' ■■ ; • 
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SHELLY 
[SHELLY . 
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OF HER t'RIP IS 
[SHOW ME HE'R TRIP-S5 J 
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- THEORY* 1 ' ' 

THE0RY#ll(3) * 
. THEORY^l-lf) . 

•THEORX#6(3) 
.V TifEORY#.7(5>^ 



THEORY7^8(7). 



"THE0RY^^9 
\..THEaRY#10(9.K 
. THE0RY#1 1(3) 
^ THE0RY#12(8)v. 



,raE0RY#13(12) 



; WHO 

[WHO 
, .'WE! 
'DO WE ! 

' ' [DO WE r • 

s 

[ WE ! ' * • • 
J DO WE HAVE ! 
_ [DO WE ^AVE .A ! 
-PAUSE-! - 



X 



[-PAUSED ■ !- 

ARE WE- \ '■ 

[DO 'WE HA-VE-Ti SURPLUS' .! . 
[DO WE HAVE A SURPLUS] ! 



.Fig. i|. Theories formed for hybrid 'strategy . 
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hytiri^/^^str-a'tegYjV excep't tov a bounded initial de\ay in* getting 



^,s,tarted,' ereems to Be a "reasonable possibility for a model of 
human: speech, understanding, since it is clear l:^uman 
prog-e.ssing of speech ^does not involve the buffering of^a complete 
sentence before understanding, begins.- The reading process, 
however, may involve more jumping around in a text and share some 
of' the characteristics of -the middle-out strategy. 

6.' Subconscious Processing of Alternatives 

in ♦Speech and fieading ' ' ^ 

Essential in all-^ of- the above' strategies is that at any 
giv^eii fime^ thereTare a number of incomplete, competing possible^ 
interpretations ,^7 requiring ,^a' strategy^ to^determine when 
processing resources should shi^ft^ f rom, , one partial theory to 
another. One might ^initially-suspeqt that a listener ' would be 
consciously aw^re of such competing p'ossible theories and that 
gifting froiri one t6 another would correspond to the. noticeable 
phenomena experienced with garden path s*entences. However, our 
experience with speech understanding systems indicates that the 
construction and evaluation of competing'"'- partial jiypotheses is 
far more prevalent than our intro*spect4ve ajvareness makes 
apparent'. It seems , then , that there must be some proc^ess for 
handling multiple alternative hypotheses that is subconscious and 
highly ' ^fficiQnt ^for perhaps a" limited class of^ phenomena)'' and 
there is some otheg.-^rocess that makes alternatives^^ visible - to 
our "perception (perhaps for a more difficult class or- phenomena 
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• •■ . - y ■ ; > 

or for those that .have not ,been frei^uent enough to, have been 
'"compiled" into our subconscious^ process) . , » 

This distinction between conscious ^ a'nd 'subconscious 
processing is of course not -an- original observation*. ' For 
example, Becker '.[ 1972] has^ referred to the former, subconscious 
processes as interm^ediate level cognition. It is likely that the 
bulk of our intelligent processing- consists of^^thlnS^ kind of 
subconscious intermediate level processing,, and that the 
"thinking"" which we are aware of at the consct^ous level is merely, 
the tipy^. the ipeberg. Con^ious orocessing seems to be larg^y 
sequential and relatively sa6w, while, the subconscious 'prbbesses 
must (by virtue of the itigarmation processing tasks that they 
acqo^j^ish) be either highly parallel or. exceptionally fast (or 
both7, ' Othec examples of the kinds of subconscious- pro<3esses to 
which I refer are the retrieval catf an^ association'* given a 
stimulus and the recognition of .a face, ' ' ^ ^ , 

"I ' will argue that the reading process ' contain's large 
components of this kind of subconscious processing, Iq general, 
a ^readar is *' aware pnly of the final Interpretation >jhich he 
places{pn a sentence in a text; he is not. aware ' o'f- ' all of. the 

- , - ^ r J' 

internied;iarte sti,ages *o^ the derivation of that^ interpretation , all 
of the- iocal ambiguities that were resolved by later context, and 
the accessing* of factual memory to evaluate plausibility of 
competing partial ' interpretations, T\Nevertheiess , a close 
examination^ of the information, processing required to arjf*lve at 
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tne^f^iixal^j^nterpretation. indicates that large amounts of '$uch 
procjessi^g ,mu^t be going on. : By way 'of exarnple., consider the., 
resolution of- the. antecedent of the prohoan "they". ' in *.the 
following pair of sentences adapted from Winograd ^^9^^]: 

a) The city council refuse<l to g'rant the women a parade 'permit 
because they feared violence* * ' * 

b) The city council-refused to- ^rant the women a parade permit 
because they advocated violence. 

\ I ' * , ♦ / 

y * * y 

This ^solution .appears to^ happen effortlessly, but the 
criteria that are needed to make the selection indicate .that a' 
very sopJiisticated • la£3reatia.l, pr^aess i^s ac^ting here belo^ jthe 
level' of awareness.-- It is not sufficient to .c^h-oose the pre/erred 
interpretations ("councir feared violence", in t+ie first ca^e and 

.- , ■ ,■ ' . ' / • " ■ * 

"wotnen^ advocated violence" irt the second) on -the basi^ of some 
simple strength of association, t^etweerf "council" and "fear" or* 
between "women" and ^".advocate". If aaything, one's a pr^xpjri 
expectatibns for such assoc-iat ions would go the o'ther way./ It 
wouJLd appear that on a priqri groun4^, women fearing violence 
would be" at least as plausi-ble as eity councils fearing violence. 
It ris only at the level where -one .begins to evaluate wl:^t would 
''"^^^be plausible groUnds for a city cbuncil to make a re/fusal' that 

: - ' • ■ ■ ' ■ ^' ' ' " 

the preference emerges. This implies* a proceiss in yWhich ^ very 
high-level ^evaluations of alternative int.erpr^itatyionf olj" the 

senteace are required before fairly low level ambiguities | are 

- - i t • A 

^ resolved . ^It does n<^t seem possible fc*,^ even/ formulat|ei the 

necessary question to^ resolve the ambiguity mlil' \ bot*h pota^ble 
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±nterpretat/ons have been forr&ulate*d and ^abomted ait least to 
the level of justif ipation of gnqunds for .'refusaLl*. It is clearly 
not the case that ,a process must* be -simple and* easy just ' because 
it happens in the head very rapicrly and without apparent effort . 
In subsequent sections we wilX consider many *more examples of 
this Ifcind of inference in reading. , ^ . ' ■ 

' 7 . Implications for PsychologiQal Mocfels of ^ Reading ^ ^ 

^ The above pifctur-e of the perception process- as involving 

matTiy competing partial hypotheses that ^are formed arid e\taluated 

* ' •' 

^ /below the level of 'introspeotiou^ has significant implications for 

the design ^ of ^ex{)erlments * €o investigate the reading prooess. 

* 

For example, it is possible that a' large percentage of trie 
, reaction- time for understanding . m||ny seat^nises is due to the 
evaluatioi^^of competing alternatives and not due at all to 
aspects 'of the correct interprefea^^ion'. Thus, two different 
sentence's or texts having' sqme identified difference" ^.n the 
structure of their ' correct interpretation ( s ) that one would rike 
to investigate may also have differences in reaction time, due * to 
extraneous -differences in the number or complexity of competing 
partial. hypqthes6s that are not^part of the final Interpretation.. 



To make .matters worse, the ^effect of .such competition 
depends critically . on ?|^h ether tjie process for handling 
alternatives is a serial or parallel process. As a simple 
'example, Qonsider a p'grallel processing control strategy Which at 
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each step- eji,t?Tids the n most likely theories! in the \v^nb qi>eue • ■ 
' for^ some' niimber n .(perhap.s - largfe)'.'^ Then^T^' tcJ '• h -different' 
theories may be 'pursued' as ' alternafciv.es 'without introducing 
^^eacfei0p time delay,. But alternatives ' i.n " excess of n 'would 
introcjuee delays-. • • , ' s . ' rr 

The' speech understanding models ..also/ predict ^tl^t a favored • 
Competitor may be found-^muoh sooner than an unfavored competitor/, 
and that the reacti-on time for a - giveti interpretation may be' 
.longer 'if the correct interpretation is delayed by.' a partial' 

e , " — . • 

theory that looks better locally but does "not. extenjj as well,„^_ : - 

In the human brain, the implementation of. such capabilities 

•may .not corr-espond to thjat of a queue sorted by priority. Other 

"implementations" could involve- a large number of active raejnory' " 

elements- that interact among themseMes^^an such a way thatVhe* 

highest priority member is selected for 'processing, .or' resources 

could be allocated to all pending* events in proportion to their 

priorities, 'etc. Thus, timing- predictions cannot be ^ mad'e 

•directly from the performance of computerized systems- running on' 

serial inachines, but.^have to be made from an -extrapolation 

such fbehavior to hypothesized mecha-ftisms in ' the /'brain . 

Unfortunately -the number or de^^^es of ' freedom" in such ' 

extrapolation is "large, ^o working" cfut a' psychologllcaUy ' 

verifiable model of the^pnocess is likely" to be, lengthy with manj^ • 
. ■ ' ' 1- ' • ■ " » " 

?.ntermediate models that have. to bfe Tarmul-ated and then rejected'.* 

- * ■ ' ■ . 
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'8^ A Close 'Look at Some Reading Material * ,4 ^ 

Having now given^ I hop6, ,^uffieietit^ reason to* Relieve that 
both *in speech' understanding andv-i^ reading ther^e 'are significant 

"Inferential^ ^qcesses •that "^^-ocycur below ; the level of 

, ' ♦ \ ^ 

introspection, and. tiiat" the characteristics -of ^ these 'processes 

are- quite different from thc^e of consciotfs, inferential' processes 
* (especially. with respect to apparent degree of . perceived 
.'difficulty versus ti>ie amount of information proces^sing actually 

done)\ I would ^.ike ' now to look-in some detail.' at the reading 

process* 4 » . 

L will present fragments fr6m * two * paasagefa^ Wi 



4f 



- - - p^asagea^ 



. s^ignif icantly daffererlt levfel§ of reading diffieulty, accompanied 

^ by a detailed analysis of somfe of the inf erent ial''..prablems that 

*must be solved. *or .their f ull understanditig . Each f raginent o^jmes 
• - , ^ ,^ * 

. from a passag'e of p§*ragraphs^ that h^is relatively self 

contained , ^ but has apparfe'lrrt:^^ been extracted from som^' larger 
work*.-' The passages are iTrom t*ie 'fRive'rside 7 Researoti . Institute 
Heading Competency Test* [.Rlversid-e ' Research *Institute , 1?74]. 

^> 8 . a . T9-ffy . ; ^ ' w , . ' - \ - ^ . . 

Taffy . i,s ' a. puppy, ^he is small.- She is soft. 
* She is svfeetn What if. TaXfr belonged to you? Would 
^ you know* what to do? ^Read- this story . ' You^ will find ' 
'^yut how^s4:4;^. take care of Tlaffyt- - ^ 

T'"^'^' Tt-ffy sleeps a lot. She needs' a _ place 't.o^^est. 

• * ^ : Take *a large ' box and cut out^ one sdde . i^t in 

• " • some 1^ hi ng ..-soft to lie on. This will^ be Taf.f y^* sAbed . 

Put it in'^a^warm place.. Keep it* dry/ an<j, cl^an , T 
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fhe apparent ' purpose of this story is to let a child knoU 

wha^t having a puppy is like.^ This includes preparing, the child 

to empathize with the puppy -and take care^of it.'" The story 

assumes knowl-edge- of what a puppy i's.. It addsk..to the concept, if 

.not already present, the attributes of smallness, softness >nd 

^weetn-ess." (Notice that . the child must . conclude' that sweet is 

no.t a taste but a personality attribute.) Prominent in the story 

are rhetorical ques^ions^^deigned to make the child think, tq 

extrapoiate from the given attributes., "^he l^tT^T'^'^ets ^ up the 

impression of iialding a small, soft puppy. Then it asks 

rhetorically^wheth^r child would know what to do with a puppy.-' 

The intention is apparently to set the child up in the imaginary 

.situation of having a' pupp.y and hot knowing what to do with it: 

'. ■■ ■ ■ ■ - ^ 

The stgry then suggests- a resolution, of thd ' diiemma — namely^ 

read the' s-tory. The result will be that child will' know how to 
,t|ke care of ai^ppy.' ' . " 



Although the entire stor:^' is' written in ' terms of a 
particular puppy, Taffy, iin:s-"clear that what the^ reader is ^o 
get .from ^it is to apply^ to all puppies. ^ How is -^^.this 
distinguished from-, say., a story $bout Lassie, in which the st^Dry 
intended to 'give at.t^-butes |of a particular ^ddg rather than 
dogs in general?.' " ; > 

The rest of , the .story, .of which the above is' only a br±l5r 
excerpt, is a conditional program for ta'Icin^care of a puppy,.- it 
iTivolves .f^rilowing ^dinections. The story mi^Kt be* rea^ for 
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general information, in which ;'case the goal-of the story ' is^ for 
the child to remember at sosfe later 'time something about 'faking 
care of puppies. It could also be read with an ^immediate need at 
hand and followed like, a cookt^qok. The following "is -a' 
sentenge-by-sentenee .account of some of -the characteristics of 

/ ' ' » 

this ^tary . - _ ' ' * ^ 

A. [ Taffy *'is*a pyppy.] ^ ' . J • 

^ This sentence occurs with no' prior context;. ' In 

V isolation,' it would be an informing statement and would 
presume that the reader knew some referent for Taffy,' buJt 
not that it was a puppy. As part of a -story, perhaps it' 
can be interpreted as introducing a character. Certaiply, 
the. intended result -of^the sentence In understandings the 
story^ is the cr>a^on ^of a temporary entitly.-pf type .puppy, 
named Taffy, ^ about which the reader expeetjSTto 'hear more in 
the course oJT the story. ^ 1 ' « 

2. [She is small.] - ' , \ ' - 

/ Thig sentence adds an attribute to the introduced 
^ .cj&aracter , (character development). The anaphoric reference 
/"^'she") here is ^asy, but- in general would be mbre 
difficlilt 'if therfe were mofe "than one potential , antecedent . 
In this case, slac^^ Taffy is at best slightly QOj?related 
f with feminine^-gender, 'the feminine pj^onoun'is more likely 

. V to 'be in6%rpreted "as adding information to the model, 
rather ;thah' a restriction on the antecedent. This, is , an 
' ^ exampi^e c f a situation whe're what wouid be a prerequisite 
in one context is adding information in another.^ ■ 

* A ^ - . ^ * ■/ 

3,4. [She is -soft. Sh.e sis sweet.] ~ , ' 

Same as above. - 

5. [What If Taffy belonged to you?] ^\ 

. . ^ fhis is a rhetorical question that exhorts the reader, 
to add a representatjLoti , of himself to' €he situation and 
draw .some forward Inferences, .(ulterior /mc^tive ' — arouse 
- feelings)' • . ' • ' \ ' - . - 



•ft.. . . 
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Another r^hetpoical question. This invites the reader. 
^ * to, ask this ^/question of himself '. (ulterror motive — 
reader .^eancludes^ he wouldn't know what to do or at ledst 
would like^some. guidance pr refreshment^ 

7. [Read this story.] . 

An impe'rative. Notice that thLs sentence' has a self 
> reference - the ahfceiJedent of ."this .story" is ^kot present 
' in the discourse., but refers^ to the' ' story itself. The 
'reader must be-able to id^ritiVy * th''e^ i^eference ♦ Note that* a 
literal interpretation (e.g*, by a .computer) might/be to, 
begin* again at the beginning to read the story', which would 
result u;Ltimately in an infinite' loop. The child is 
presumed,, to be sensible enough to Icriow that he has already 
read part of it and can f ulf i^l ' tfie commaqd by merely 
reading the rest of the story. J Alternatively , he can 
"assume the phrase "this st*ory" is *, a forward pointing 
reference- to what remains, and- that T:he part that- has just^ 
been read is somehow a preamble^ or* something,, but not 
properly part of the^ story. The child could conceivably! 
-^Cw take either of these alternatives., ^ 

8. [You will find out how^ to take care oPTaf-fy'. ] " - 

A declarative sentence, but presumably interpreted as 
the consequlpt of a conditional^^ — "you will find" out ... 
...if you read the story". -Not only th.at, but it says that. 
. you »>wiai learrt hcJw Ho take care of Taffy, when in fact/ 
what you will learn how tv Ho is taj^e carfe of any' puppy. • 
In the ' entir'e story, Taffy is \clfearly the name of a 
variable t.H^t can later be instantiated. Does the chj,ld. 
. f^iilly undei?stand this? ^ Does anything ixi the subsequent 
• story depejfid on Taffy being ' female?^ How does^ the -child 
'./*"->det ermine-** at what 'level to generalize what he learns.'' 
E.g., to all puppies? to female puppies? to 3mall, ' soft, - 
female puppies? or -i^erhapp' to small*, soft, Animals? On the 
,^ other, .hand , if he were really Jeaf?ning ijow to take c^re ^of 
-Taffcy, a particular dog; he' would have to know where she 
- liveH, etc. jSL ' — , ^ 



Aft^i^ the •> firat paragraph, therfe is an ex^ct.ation that a major 
portion^ of the rest of the .story- (maybe^ all" of it) will ^'be 
irTstructions '■^ in how.^ to take care»of a ddg.' We will discuss Just"* 
a few more sentences. • , . - • , - . 
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9. [Taf/y sleeps .a lot'. ] • = • " , ^ 

. • • ♦ ~" 

Thrs adds" m'ore .to properties of a puDpy (Taffv'')' 

pailgra?h) ---motivates «ext. , senten^fLd .Lt'of 

10. [She needs a place fo rest.-] ■ ' ' . '■ - 

• ~ . An implicit "therefore" relates^ this sentence to the 
previous one. (ulterior motive - • crea-^eS a need.vhllh ' 

• .subsequent, sentences will tell h6w to meet). Does a 'cK?S 
.pick up the implied causality connection between this and* 

the previous .sentence? . - • . 

[Take a large box and cut out one side.]' 

^Thisj imperative is n-ot necessarily' to be^djone 
immediately .as ^ was the ^one in the first paragraph. irSxl 
7.lf%€ ifde, he might do it imn^ediately but 
?n?o ; '^hT" ^^ better tQ incorporate the • instruction 

^ns?nuc?tn^;. k'/' ''"f? '^"^^^'"P" go on to read -more 

instnuctions before doing .anything. If in the mode of 
reading for future, information , the reader should form the 
plan,- and also, elaborate the reasons for- the individual 
steps, so that he will be more likely^ to remember it The 
previous sentences motivating this* sentence help in-^this 
respect, but the rfeader .should^lso internally anSeJ such 

SSh^tT'r ""1?^ "Why ^"^■^'^t one side?" , and 

"What am I going to do with -the box?^' = Also, he should 
^ understand that this use of "take" does not mean "steaS ■ 
Likewise, he, needs a decision o^ how large is large"? This is 
•one reason for in-corporatirng-.-the . instruction into a 
tentative plan and. ■ then going an to elaborate- -.the plan. 
Subsequent -decisiofis^may have a" bearing on how large the box " 
must be. For example, a r.efrigerator. box or piano box - Is 
not necessary.. • , - •• . *^ . 

8.b. Growth 

- Jt^^\}'^\. characterized by organization. = Group '9 
growth charts show many age- level uniformities - and 
°i }^ -f® ^^^^^ changes^. Differences between 
f^^l'^^^^ ir^^growth certainly exist. However, most 
injivi^tials tend to be only slightly variable in their ^ 
,^ate ^of growth from one age level to another. ' 

Th^s 'story^is much mpre abstract 'and so'phisticated than the 
Taffy story. If 13 apparently intended to impart .to the reader 
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some general 'information m^bout 'the growth process -for**, some 

indef/inite future use. The passage could possibly be an excerpt 

from a biological t^xt book. ^ it ^is character!/^ by a much 

greater^ . necjgssity for the .reade<; to sus,pend judgment on. possible 

int.erpretatioris of portions of. the- passag-e until *'he has read 

further.' Among other things, * it is' characterized by a large 

nupber of noun-^noun modification constructions '('^roup growth 

charts" ,^"age level unifc5rmities" , etc.), where noun>is^used to 

modify another noun. The intended maarii^ng of such constru<?tions 

*is critically degendent on world ^ knowledge of the habitual 

rielationships between t^he two nouns. ./A detailed account of some' 

of the diffici^ies of' this passage follows: , • * . 

J. [Growth is characterized by organization.] 

^This seatena^ introduces the topic "growth" and focuses 
on its organizational aspect. It doesn't, say much as a 
factual* assertion . .If the' reader asks hiiaself -a great -deal 
about this question out of context, he may try to ir^stantiate 
examples of kinds of \ growth -and try to find instances of 
organlzatfon as'sdoiated with* them. In all likelihood, 
however, ^ he will do little with thi's sehtence, , but go on to 
see what's- cpming next. 



I, 



2. [Group growth charts show .many 'kge- level Uniformities and 
predictable age level ' changes . ] ^ " - , 

'This is a d^^ficult senti^ence •syntactically. It has 
three noyn-noun modifi'ter ^ constructions* which • require 
-^inference t^o • (determine, their meaning.^ If the phrase "group 
growth charts" iBeans something* to the reader as a . technical 
term iri s^ome fieM^, then the sentence is considerably easier, 
but otherwise, ne^ is"^ left \ wondering '^groups of' what?" 
.(pbssjlbly people', but .the story 'doesn't say). The \most 
general possible . assumption is growth of organisms (or 
perhaps even mor'e general , still .including ^tnstitutibns , 
cultures ,* empires^ etc. - all of this^ i^i^oon-Slstent with the 
very abstract first statement, 'and it is possible that the* 
• " vagueness* and aj)stractness of the first'.sent.ence is inteli'ded 
to make such* jjenera^ interpretations^ of the^second possible'.) 
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The phrases "age l^evel ^bnifprrai ties" and ^'^1^ level 
changes" are again considerably more easy to understand if 
they are previously known technical phrases than, if they have 
to 'be figured out from scratch/ In the latter case, the 
reader needs to decide whether the phrases use the word "age"' 
to Dio'ddfy the phrases "level uniformities" and "l^vel 
changes" or^ whether the' phrase . "age level" modifies 
"uniformities" and " "changes . " In this case, we assume that" 
the reader is familiar with the concept of "age" ley-el" and 
wduld ' for, th%t reason choose the latter interpretation, since 
tie 13 not likely to find , a .plausible reference for the phrase 
"level uniformities". (Note that "J^evel changes" -is a 
perfectly good phrase, but would require .a subquestion 
"what kinds of levels"). The senten<?es would be more 

-^roperl-y punctua^ted a'3 "agerlev,el uniformities" and 
"age-level 'changes", but such punotuation is frequently 
nlissing and. a good , reader is expected to be able, 'to 
understand the phrases. ^ anyway . The difficulty Is, greater 
without the hyphehs, sinc-e it opens up the possibility of 
different, interpretations ^^rid requ^ires the reader ^to resolve 
thie ambiguity using semanl^ic- a<id pt^agraatic considerations, . 
< ^ 

^[pif ferences between individuals in growth -certainly-- exist.,] 

\. It ' s^-har'r7to .say what this sentence does, except^to set 
up^ expectatipn^'*: for what jwill come next. The choice of the 

-word " "individuals" suggests a' rest.rictipn of -the .notion of 
growth bei,ng talked about to single organisms and perhaps* to 
people, rather thaa* the more "keeping generalizatio-ns -to 
organizations and cultures that were po^siMe for-sentencjes 1 
and 2. For a reader who jumped t'o th^ cpn^ailusit^Ti that ^gro^/tjh 
of .people was being, talked about at trie J outset , ' this s'torV 
may actually be easier than for trie more int^llectiHal- reader 
who made.»the mbre^general extrapolation and is now/ faced wit^ji 
the necessity to-reconcile this *use of the wor^d "individuals'! 
witH his -previoiMS generalization. 

It might.,.±>e* possible that this sentence^Just referr*ed to 
certain kinds^of growth (i.e., of -^individuals),' while the 
story as a whole i/ould S^still be ' aboutT the %one genecal 
concept of. growth. However^s^ome pragmatic rule seeraa^vto jgay 
that if that ^ Were the case', then the story 'would "^haVe 
cor^tained some transition indicating a temporary shift to 
considering growth ^bf individuals. Such .a ^ sentenc^^^^ght. 
have been "In individuals, differences in growth .exist" .(with ' 
a presupposition that what is being said' is not true for 
growth in other than individuals). In the aj>sence of such*' a 
focus-shifting locution, the good reader^ should assume that- 
,t^his^ sentence i's still about the s^ame , t"bpic at the same 
level, and 'therefore his ^ initial' uncertainty 'about the 
generality of the concept of grqwth.^ \being disQuss^ should 
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here be resolved'^ to refer ; to the growth, of inciividuals 
(probably human) • ^ * \ ' ' 

Some processing load, whicW may-- have been considerable, 
- was required to keep this decision -open to this point, and 
some IB still requirecl' until the reader de^^^rmines whetfier 
the .individuals of, concern " are ' people gv individual 
organisms. The resource expenditure requirecl to keep, ,such 
, decisions pending m&y. be considerable, and may significantly 
affect reading diffioulty or the performance of resource 
limited individuals in'reading such piassages. - ' 

4.. [However, most individuals tend to be only §Iightly ^feriable' 
in their rate of,^ growth^ -from one age'^ level to another..]' 

^ * " ^"'^^ * 

The ."however" 'here has the'^'effeot of negating or 
diminishing ^the strength of the preceding.' sentence. The 
general pragmatic rule seems to be that "however" will 
introducer a sentence (or paragraph) that contradicts some 
expectation^ that the reader might bthe>wise^' have drawn from, 
the' preceding sentence. (or paragraph). It fakes- considerable 
inferen-ce to determine .which specific expectation from the 
previous sentence^ is here being ' -contradicted . One 
possibility is that the author is trying to contradict the 
r inference *that ,the differences referred t;o 4n* the ' precedcijig 
s^en'tence- are: large*. (This would explain the^-^^^^.of 
"certainly" in the preceding sentence^ although I anTnWWre 
what -the rule is, or whether the use of "certainly" Jladsft^ie 
reader to expect something of thj^s sort to'' follow) . ' 

TJ&e use of "moBt*'*^ implies that ' not ^all individuals 
satisfy the assertion. The use, of "tend" - lsj> anotlier hedge*^ 
word,, that says the statement- is" npt always* tr.ue. The 
statement so qualified, that, is* the ' predicate whi^shr tends to 
be true qf.mqst individuals, is that their growth rate is 
reiatively^r stabie^vfrem year to year. Thus, "however" is now 
. explained not as C:0>itt*adicting the pQssible inference , that 
the differences are, large, but instead it contradicts the 
possibler'inf^rence that differences in one (Jimension (from 



*|iote that 'I -am ^not necessarily assuming that the^^ reader 
a|0sciGusly invokes ^ models of the author and his indentions in. 
understanding .such sentehcesj the riales can be formulated in such 

- a way -that .no author or ulterior^ motives need to be explicitly 
men'tioned. -'However', it is* probable that -a reader who dofes 'have ^ 
specific model of authors and their intents is be^tter able to 
understand such passages; .by /invoking such a model, as discussed 
ix^ Bruee . [ 1978]. It is likely that the necessary rules can be 

'more succinctly rejiresented and learned by lising ttee concept^ -of 
an author and his intentions, than -by ^learning them by rotQ,., . ' 
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individual^ to individual), might indicate differences in 
anoth'er dimension (from year .to year). To fully understand 
these two sentences and their * uses _of "certaiply" and 
"however", the reader -must have a concept of this kind of 
* cpmparison . ^ / ' a . 

Whether it i.^ necessary for the reader to actually set 
, up a' model of something potentially varying over several 

• , dimensions in some space, and mapping age differences and 

individuation onto two - such' dimensions, I am not sure. 
How^^ef* , it ^ is_ pl'ear that a reader must eitKer ignore' the 
cues giv^ri^by the use of "certainly" and "howev.er"; in which 
ca3j h6 risks 'misinterpreting some part of the story, or else 
Ffe ynjust have a sufficiently ^rich analysis of the sentences 
and (their possible interpretations that he can fully account 
for them. A^reader who is -in between these extremes, who is 
^trying to account for the use of the words, but is not adeVt 
at constructing possible underlying models to -account for 
them^'-Will find this' passage more difficult than ei^ther of 
. , 'the -other "two' kinds ~oT readers. ' It would be a considerable 

• challenge to devisje a test that would distinguish these three 
different levels of reading. One complicating Cactor is that 
which of the three levels of comRetence a reader haa mastered' 

» would probably vary considerably from one set of syntactic 
cues to another ^ ^ - ' 



Nptice that the purpose of this story .seems ^to be to build 
up a fairly abstract mod'el of how growth occurs., Thg. reader is 
not asked to do ^nything immediately except som'ehow to assimilate 
t^e ^ story, ^ The purpose of the* reader in- reading the " story, thus 
becomes of paramount importance, and if his 'purpose is to prepare 
himself to perform in some way in the- future usijig a model- of how 
growth occurs, then .he is likely" to\ 4o different things in 
reading.^ the story than if his purpose is, say,^'1:o be preparfed to 
answer questions at the end of the passage^, or^to fill in missing 
blanks in the story. OT^course , df this passage is read by an 
elementary school reader,^ it is likely that he does not have the 
goal^ to prepare himself 'for some specific future performance, 
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since it is likely that he cannot even imagine a -specific use for 
the information. Howev?r, the author may have, such a use in miin^ 
in-writing it. In particular, the author might be writing for an 
adolescent who may be 'concerned th^t his own rate of^'gi^wth is 
too ^slow or .too fast, and may want to convey such a ^growth "model 
so- that the .reader can. understand ^hat ' s- happening to' him. It 
seems likely that two students, one of whom is concerned about 
his slow rat:e of \;^wth, .^nd another who is determined .to .become 
a biologist, will get colupletely different things .out o-f this 
sUory. , ^ 



Some Details of ^the Reading Process 



The previous discussion has outlined ifme of the inferences 
tha't that need ta^be made in 'the course of' understanding these 
two passa^s. ' However, it sheds * little ^ light on. how those 
inferences might be organize^ and carried out. Experience With 
computerized . language 'understanding programs -and speech 
understanding systems can'-give \xs some insight into how these 
processes' might happen, "although^ the picture at" the moment is far 
from complete. In the remainder, of this paper I will 'discuss in" 
, more detail some of the low level" decisi'ons that ^have to be" made 
and alternative hypothese's that have to be generated and 
ctinsideretl , pointing out as I do so where existing techniques in 
natural language - processing by computers have been developed to 
handle similar problems. I will consider in particular the first 
3 • two sentences of the Growth passage. / ' 

~ . ' ' '/•.■* •• 
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The first - sentence^ "Growtn . is characterized by 
-organization," Is no't difficult syntactically. . It ' is a 
straightforward passive sentence with little poteptial for 
syntactic -ambiguity ^ ' The first word/ c'an - only be a noun, the 
^second is unambiguously a verb' (although it^ is not clear after 
only two w5rds whether it is the main verb of a copulat* sentence 
or an auxiliary verb 6^ 9 passive)'. The f^ct that the third word 
is a past participle resolves the sense of "is" to"ttiat of an 
auxiliary in a passive sentence. [Woods, 1970] gives a' detailed 
account of how ATN grammars can- be used to recognize and 
disambiguate this main-yerb7auxiLiary distinction . After this, 
"by" is^_unambiguously a- preposition (although whether it is 
indicating the agent of the~ action or introducing something which 
an -action takes place beside is not unambiguous). Organization 
is unambiguously a noun. Hence the discovery of the syntactic 
structure of this sentence is .straightforward and involves little 
nondeterminisml^. Every local "^yntact^c ambiguity is resplved by 
the immediately followiTrg-^word . . . 

What -the sentence means, however, is something else again. 
. Both noun phrases in ithis sentence are mass nouns fnouns that c^n 
be used without ^determinef^s in -the singular form as if they 
) deriote'd a substance,' as opposed to count nouns which "can be- 
^ounted). When used in the singular with n^d^terminer , they are 
to be interpreted as refering to the general di^ncepts. that they 
^ame • , ("A ' growth", . "groi^ths", <or "an organiizat ion" ^/ould get 
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completely different interpretation ; the. difference is' flagged 

/ * 

by the way the words are used syntactically.) * 

"Growth"-, as was pointed/|oyt -previously, can name several 
.different, concepts that^ the readifer might have in his head. If he 
has only one such concept, thin '^the .readihg task "is "easier 
■'(although ^- it may get difficult later on .if the concept he has is 
not the one the author intended). If he has ■ seyeral, then the 
S interpretation of the sentlnce ..is semantically ambiguous with 
respect to thie r^eference of |his phrase. This can be represented 
temporarily by associating | with ' the noun ph/ase a list of 

alternative pbssible' interpretations / (such as is done "in the 

' // ' ' ' ' ^ 

semantic interpretation procedure of the LUpR system [Wopds, 

1973a]) . 

- The interpretation ,t>f "organization" appears to be somewhat 

different, Largely because of its absence from -surface subject 

position in * the-; so/ntence. (Notice, thai "Organization 

characterizes" growth" wbuld" not have the same, effect as the 

> initial s^^nt.ence ' of this passage.)' The correct intet'pretation o^ 

the sentence is' an/ assertion about growth and what^'is being 

asserted is. that it hks many -of the, properties associated with 

' / ** ♦ 

organization. That/ is, ■ Porganization" / names a concept where 

certain cha'racterisyics are to be found that^are to be associate(J 
now with'growth. 
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The differences In interpretation of^^t h^ tj^ y noun pljrases 

^are^due to their position 4n context as different arguments pf 

the verb "characterize" and :as different role^^ fillers in the 

surface? structure of the sentence/ The LUNAR system handles such 

differences in,, interpretation as^ a function of context by 

providing . context cjependent parameters to the rbu'tine that 

computes possible interpretations. of :constituents\ These 

•> * ^ ' 

parameters are used to determine the interpr^ration rules, to be 

used for interpreting a constituent. .(It is possible in general 

for different possible ^higher interpretations to- call for 'the 

interpretation vOf.. a consSi^uent in different wayS| so provisions 

are required to keep^ alternatfve interpretations of "different 

constituents coordinated with each other.)- ^ 

for interpreting the 'clause as a whc/le, a semantic pat^tern 
for "characterize" (or- perhaps '•for "is characterized by") is 
requice^J. ^ This is a fairly abstra^st^arnd somewhajb vague semantic 
relation. My edition of Webstei/^s gives two ' senses' of* 
"characterize" -•- (1) to describe the character or quality of: 
DELINEATE, and (2) to be a eharacterist;ic of: DISTINGUISH. - 
Leaving .aside the problems of the adequacy of ^uch dictionary 
definitions, I interpret these def initions'to focus on two senses., 
of characterize — one in which sufficient ^conditions '^ are given 
as ^ in "four equal length sides and right angles characterize a 
square", and the other ih whidh merely prominent .characteristics 
or only ^sbme characteristics are ^given. Which of thes*e two 
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senses is^hosen'- would make a big difference in what * the Velder 
believed .the passage to say. In the first case, it wouf^li say 
that anything'' that vas organized w.o,uld be growth obviously 
false if one Icnows- anything about 'growth .and organization^' ThV 
second- iliterpretation is that ' organization ^ • is a '' prominent 
characteristic of _ growth. The reader must decide- Whether this 
sentence is trying to inform him of an astounding' new fact or is 
merely asserting "organization as a property of groilth. 
Presumably the- second choice is more plausible than- the formW" 

■■ ^ . • . : • ■■ . .. ■ '-S^-^ 

In the LUNAR system . such ambiguity, of ' vord -sense was- 
indicated by having several sera^iq^c interpretation- rules' 
associated with a given head word, both of which- might match a 
given . constituent being interpreted. Thus .a ; proc'edure- for 
generating both possible interpret'a^tions is . straightforward . ' The 
.^r^oblem of evaluating wjiich^ pf two .interpretations is mor,e' 
0iausib'le is more difficult, and no 'computer system^at thS momen't 
makes such plauslbilitylevaluations . Almost all current computet, 
models of such processes make all-or-nothing decisions that an'' 
interpretation Is either, possible orf Impossible , with no shades 
in between.. ' ■•^-r-- r - - ^ . ^ . 

^ One is ten^ted to .say that the above .choice is 6l)v1?ous (and 
implicitly therefore easy) , bu^t^it is not clear exactly How far 
each of- the two alternative interpretations has to be 'elaborated, 
befone the choice can',, be made. It seems to be necessary tb 
ac'tually" formulate the erroneous interpretation and pose' it as a 
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^ questipri against one's knowledge in order to det'ermlfte 'that' it- is 
false and therefor'^ the other Interpretation is 'to' be -preferred . 

'.. Consider :herQ.tiie understanding task -imposed* pi^ a' reader who 

was not_ aware . of -both sens^ of the verb'^"chara6teriz^". -' jf he 

•h^d only, the c6rre'ct interpr^S^ion then the task would in fdfet 

. be.^easrer than for a raorfe advanced' reader -who knew them, botii.- On" 

the' other 'hand if .he'^'had only ttje Wrong sense, tften. the first: 

... • ■ ■* ^ if ^Nh* 

sentence, would bW apparently false, anc^he would ' have% • dirficult 

time with the passage^ 'An additional -possfb^ity' is 'that the 
reader d'oesn't really ufiderstand, wLt ""characterize" .'means' and 
iwt^rprets this Sentence'-meV'ely as 'establishing some .kind of 
assocaation between gnowth/and orga-nization>, ' which, as it turns 
out, is abqu^.all that the >entenc'e' -is' intended txs' accomplish 
anyway. The primary role of this first . sentence ■ seems to-be 
.merely', to establish growth as "the ' .'topic and. perhaps bring 
..certain aspects of 'gr^th :^nto focus '(the ' :!<QrganizatioRl 
•a-spects) . t;ius , ■ the " reader ' c^n ge-t the a.ppr^riiate "ef fectV 
without fully unucieirst^nding -the-^seritenc-e a't all ' ' " ' 

.yThe secQnd sentefice, "group gro^tl charts' show jnany-.age. 
level.; unif^orjnities and predictable age -level Changes?', is faY- 
jnare complex, syntactically ' as . weir as skanticaLly. English 
s>rttax perm„its -tiie use of. 'nouns ta -modify. either noi^i^/lTTlmosf 
infinifc,e-pnofusion, -but the ilieVpret/tion of the meaning, i^fepends - 
on non^syntactic world knowle'dge". Moreover when more than -one 
such^- noun ■?iiodlfi^t. is used; i "st^ructur^r syntactic ,amblgulty is 

introduced that r^gulreb^r^l'd know'lpdge to resolve. ' ^ - . ' , 

■ ■ ■ - 39 r • . . r . . / 
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^If '"group growth" occurred in isolation the a^tructoire ' of a 

noufi^ "group's modifying'* another noun, "growth", would.be the 

pnly possible- 3yn^ac'tic interpretation! The determination ' of 

0 ■ - * ' \ . • * ' \ ' 

^what it means ; would depend on - the ^. ability af the rea?ier to 

ident:U'y a plausible relation between the two words . (in this 

- ' w , 

case a^grpup of things can grow, ,givxng rise 60 arr interpretation 
.^^♦growth g[f- a group"): However, in^/'gro,4p^growth ^charts" it is 
ifl^l^iguous .whether "group^ modifies^ "growth ^ charts" gr ' "group 
growth." modifies "charts". • ' ^ 

The <» fatjt tlrat charts are de^ic.es for deplc^ir^- things give's 
♦"growth charts" the ^possible Interpretation "chart's ^depicting 
^g.rowth",-. and S''grbup growth charts" the possibly iateppretation 
"chants depicting the growthlof groups". ' However, the correct 
3,nterpretatioh is probably "charts depicting growth by 'group'! 
de,riv^^rom; "group"'' modif/ing "growth charts", requiring Jfche 
reader to either know about «s^or imagine a kind of grovf-th chaft 
that would be distinguxsliat^e by: having something to ' do with 
group^s/ Jhis meaning • Ts not * very . distinct 'from one of the'^ 
possible ^irit^pretations of "charts depicting growth of groups" 
(indicating that • tbe mea^g -o^^that strucfure is far fron/ 
, unrque),'and there is certainly not* enough evidence at this point 
in the^^-sS'lgQ to re3blve which . 'of these different syntactic' 
structures- or which ^of their semantic interpretations is to» be' 
taken as the author ' s..intent4ipjL. The possibility of this ph^se 
havirig any of several possible interpretations^ needs to.be held 
open .until more of *the sentence is processed. 
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V 'Thfe process of deciding that "charts", is 'the l&st --word in 
the noun phrase "'and that "show" is ma?'{nv^rb requires some 
further -local -ambiguity," althoufe'h the*" -faist that there is no 
determiner On the noun p5hrase requires' that it end either with a 
plural noun or a mass noun, so that "show" could ohly be included 
if it were followed by. another noun". <^*Shdw" cou3.d be ,a noun 
\instead-of a verb, although it would be. difficult to put*' a 
|plaugible interpretation on "group growth cl\arts show" as^a"^ noun 
phrase in this context.") ' 

Exactly, sinlilair problems are encountered in interpreting 
.'•age level..-unifornlities" and' "prediptable age ^ level changes". 
The latter- has even greater potentTal^ ' aipbiguity due to' tfre^ 
possibility q,s. of "predictable", modifying "age", "level"", or 
"changes" (p.g'. , "pr?d.icta"ble_ changes in, age level", "changes in 
thC_predictable ^ age level", or ^''thanges in 'the level of 
'predictat^ie age").";. Somehow a r,eader. makes a choice from ambn# 
these. different possible interpretations,' usually without much 
coascioj^^ . effort and usually ■ -correctly or at least- one of 
several equally .acceptable"^ interpretations) . In^'this case, none 
of the above possibilities is correct , ^but^Lnstead , the. thing- 
that-i*s doing t^Jt^anginig is eHiptical Cpres-umably - growth or 
some -growth rparamet'e'r such) a^, height/oV /.weight ) and the correct 
interpretation is more' like- "predictable changes [in 'some growth. 
pararaeter]£as a function _ of ] age level". Changes "in a growth 
par^eter as a f'unctijon of age are certainly to be- expected and 
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therefore "predictable'^ whereas it/ is difficult to imagine 

anything pj:i a growth chart vthat ' would ^correspond toi a ^predictable 

'age^ level (yhat , would ^be -doing the'^ ^ predicting) . 'f3feS||^ a 

probable role, of "predictable" • is'* to mo'difythe concept "age 

level changes" as a^'whole, although^probably as a.jion-restrictive 

modifier. " • * . / ' ^ ' 

The above\interpretation as nipt completely corr^ect, since if " 
a similar evaluation of "age leyei^uniformities" is carried- out , - 
one *^^T5 ' led to^j^look for something thai is uniform within an age 
level (again, presumably some grovfth parameter) The ^ac.t ^th'at 
conjunctions require HOme/^degree of parallelism between the two < 
things being;conjoined ^Jjpears "to de'iiiajnd' that the role, of . ^fie 
phrase "age levell' should be "the same- in the two co^hj'tinct©, ^This 
slightly^,jcontradicts the oj^herwise well-tnotivated in't>e,^retatlon 
of "predictable- age ^ level ^chahges^^]^ above'. ^: ";-InStead| an/ 
inte^^p-retation-- in which -something . changes .within an age level 
rather than ag a funbtlon of - age level is veq^lt^edT to=maintain 
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•this parallelism. This can^ be met by repl^^ the relation "as 
a f unction of^'age level" that ^as p6stu^ai|d-. aboye- ' 'with''- "wilhin. 
'an age level". This,* however, nemovfe'^Hhe' Mundation for Ihe 
argument justifying "predictable"., fio^olcing 'Vii'rther ' for a 
different jugtif icatioa, onp can supple that the eharte somehow 
^ show predictable changes (i.e.^ -the charts do ,ti}e prediction?), 
and that "predictable" • is. a , restrictive adjective here telling 
something about the\inds^ of changes that the charts show., V 
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Computerizetd parsing " aagprithms using formal grammars for 
extensive sabsets of English \5an,^ystemalically enumerate' all of 
the possible ways of grbupingth^ words ; in such noun-noun 
modifiers^ bytJ- the process is 'usually combinatorically expensive 
and few. computer* models' deal with 'such ponstructrons . Attempts 
to- use semantic*" information ,to guide a parser to construct only 
possible interpretations havjs been attempted, but nothing s that 
begins to match the complexity of the above discussion has 
currentlj^ been implemented. In general, the techniques for 
efficiently _coupling syntactic and" semantic knowledge, in ^uch 
situations are still being explored and the results are 'not in. 

The di£rcove^y of the preferred, interpretation in the above 
example requires the judgment' that all of the various- ways -of 
•gro\iping theyndiyidua.1 words in this noun-noun modifier sequence 
have^ implaus-ible' "interpretations .in this context~^( what a &roup 
growth chart w6uld show:) without the addition of an' additional 
elliptical participant in tl^ec^under lying rep^sentation (namely* 
what changes).. The chain of r.eas'(5ning justifying the 'correct, 
choice is something like: 

. "I knowpthat a gravfth chart should show changes in 
some growth parameteF i Wi^t ♦ s what growth is) and not* 
changes, in growth -(at/rii^t not directly ,•— thatn§, the °" ' 
derivative of what''^ a ^growth chart would defict), so 
that must fill the "ch'angee" • role* of the change being 
.discussed.' t recognize "age level" as a concept, «so - 
_that is pV'obabiy the role 'that "age" is filling and not ' 
L a modifier of "level changes". ("level changes" is 
such, a concept also, but 1 can find" li plausible' 
connection between ""ag.? level" and "changes" and" cannot 
find one, or at least not a 'different one, between'' 
. "age" and. "level-changes"). To relate "age leveP' to 
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"changes [in^ growth parametefj"^ I can, take^advaijtage 
of further knowledge (or imagination) about growth - 
' -<5harts^\^ao'd . speoulate ^ that tKe ^charts might' be broken 
down by "age level". That couid be* the role that* "'age ' 
level" , is ' filling, her^^e^ - ^but that would, violate" 
parallelism of conjunction wii^"* the previous phrase, ' I 
could either, try to reiijf^cpret^ the previous •phrase to' 
establish pa^rallelism^ ^dr I can -try the same relation ^ 
between "age level"^ and "ch^anges" that I used ^ith 
, "uniformities" before. The latter works, so lUl try 
that. Finally, "predictable" -must modify "changes" 
rather than ^^age" or "age level", since I qan imagine 
charts somehow predicting changes more easily than 
their predicting ages." ' y , • - ^ . 

* * ' 

Making this justification, as complicated as it seems, is 

relatively e^asy compared to the steps that were required to - find 

it amo-ng all* the other^ poasibilities - formulating alternatives, 

< — » 

making negative evaluations of. some - of them' and differential 

choices among others, aad finally settling on a xjhosen 

interpretation or. several likely onesu Clea-rly some readers may 

boggle in the face of such a pj|sage and give ^up. Others may 

h^ve the processing capacity to carry " through the' kind pf 

analysis outlined here^ , either entirely or , partially below the 

level -^f introspection. Still others may adopt^pme control 

strategy that does not con3ider all .of the alternatives, -"Some of 

them "toay do so with srtrategies that are still likely' to obtain 

the correct interpretation most of the time (assuming that there 

is^ one) , while others may adopt lerroneous strategies that clt)om 

them to misunderstanding . ( Even these latter may^ find - 

"i^erpretations that are intellea,tually satisfying to themselves, 

causing them to assert^tbat they understood the passage, although 

what tiiey have understood ra^^be almost totally unrelated to what * 

the 'passage -says, ^ , . ' ^ 
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To close our\discussion of the interpretation -of tjhls second 
sentence,, let me- point out that a lot of the reasoning that 'was 
used forv selecting an intende,d^ interpretation for "i)rea4,c table 

mded 



age - level changes" depends oxi ' th^ fact that "the J 
interpretation must fit ^the context "group growth charts show 
..." in -a .passage^ whos^ topic is' known to' be' growth (of 
something). Effectively, many dif ferent*possible interpretations 
had to be hypothesized and^ tried^in this conteoct for a possible 



fj-t.^t appears that below the level of cbnsjcious awareness,- a 
gregit-deal more hypothesis enumeration ^and evaluattdn is going on 
than- one would first suspect. Current computer models^end to be 
based on . -the. assumption that * the amount of ^Uch hypothesis 
formation can' be ^controlled by having the right dominating 
"fpame" or "script", sinae' the .cost:, of considering many 

* * * ^ ' 

alternative hypothese's ori a^'serial computer ^ is -prohibitive. I 
suspect, however that this-^is one of the . differences between tThe 
Serbia! computer and th,e human brain that is significant and that 
in this i?espect the characteristics of computers as models of 
human processing is misleading. Much of AI work wilX^ continue to 
be 'focussed in this direction because of the des^irability .of 
getting computers to do such tasks, but I suspect -^-^that valid* 
models of 'human performance^ will include much more parallel 
evaluation of 'alternative hypothecs. 
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10.. Goncluslon ^ < ^ " 

■ • •■ ' ■ ■■' ' ' . ' 

The above discussion does not' go ' into'^d-etail a^t the level of 
recog'ftition of individual words and letters , -.'but one ey^n model.' 
\^them with similar pr^ocesses, so ttrat the overall .reading process 
-i^- a cascade of levels, each of whl6h is.Vaftng only 'tentative 
decisions. Each . level will be 'formulating^ many alternative 
hypotheses that are to be partially selected by virtue .of the 
degree to which they are compatible vdth hypotheses-at t)tiier 
levels. For example, the syntactic compc/nent in the BBN ^speech 
understanding system makes ; many alfer^til^e hypotheses about 
..possible syntactic- paths -through «ach of the theories that it "is ' 
-given by the control component to consider. Rumelhart [1977] 
gives a^sketch of such -a multi-level model based on the 
hypothesis structure of the Hearsay II speech ^understanding ' 
system [Leaser et al . ^ 1975]. 

, ."I- -have tDd^ed tc^^give at a fairly ^concrete level' some picture 

of the hypotjiesis formation and evaluation processes that must go 

on during reading although we are not normally aware of them. 

^ . ManV of these processes -include inferences involving the kinds of 

metaknowledge discussed 'by A. Brown [ 1978], although"" " the 

. . presentation , here puts those processes in a somewhat different 

light. Here, ,1 would stress that although we h?ive some knowledge 

about what 'we know and how we know it, this know.iedge is based on 
— _ ' , " • - <• ' 

-1 introspective observation in much the same way th§t our knowledge 

" , °^ any.aspect^of the world is based on observation. .We do not in 

• ■ . < 
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general have any privileged access to sonre "internal truth" ih" 
this respect. We do have' an awareness of certain internal mental 
events _thdt are, „ pe-rceivable in-^^ much the same way that bur 
external sense organs perceive the -/worl^ but they do not give us 
"a complete awareness of our ir^ernal mental, processing, and we 
have t^ learn the significance oX what awareness they, do give us. 
This is^ presumably^ why the metacognitive abilities come rather 
late in the s-tage of mental development. 

, Experiments repeatedly show that what appears to be memory 
is in fact reconstruction and that many "memories" ' about whTch we 
are ^, absolutely certain turn out to be- mistaken. Thu's^our 
metaknowledge of what we knowand how/we know it is only as ^ %dod 
as the model 'we have built up -based on' our "observations of jbur . 
own performance and cannot generally be relied on a^ absolute 
truth. Conversely, a correct understanding at the irfeta levll of 
how we should go about some mental process does not automatically 
translate, into an ability torcarry out 'that process. 'The attempt- 
to consciously follow a set of instructions is not the same as 
fully incorporating those instructions -^nto o~rre»s internal 
prbc^ures.^ Examples of, this phenomenon abound in such processes 
as leaVning to drive, learning t,o sail ,' learning to 'play , chess , 
to /solve mathematical problems, etc. The process whereby 
related • attempts to^. ^follow such instructions eventjjally 
"compiles" an internal_ procedure for doing the*task dnd the means 
whereby aonscious resolution, to "do it different next time" 
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actually modifies such .procedures are almost totally mystferious. 
It. i-s the attempt h modeling, such ( non-introspectable ) 
■procedures by .computer programs and ^strac^ automata' that I 
believe holds the_ Key to ^understanding them, and it is this 
understanding which is 'the 'key to effective. edu^atioVial 
strategies. " * . . ' ^ 



1 ^ will not pretend that jthe results - «f " - Artff icial 
Intelligence and Natural Language Processing research to date can 
give ^ complete 1 account of the processes outlined ^ above 
However, they do provide- a very> rich inventory of analogies out 
of which one can construct hypothe-tical "brain cojnputer 
architectures" and reasoning' strategies \that might model the 
information processing operations that - go*, orf. in" reading. The 
major contribution whicH the AI appro-ach has to offer, I thinkV 
is that it reveals' and makes concrete information processing 
steps that one might ..not otherwise have suspected." It can thus 
serve a- very valuable function iri - the investigation ' of - the 
reading pro&e^s. - On the other hand^ one .has. to be,. careful in 
extrapolating results fr(^m^ computer models to' humah processes, 
since certain characteristics of any computer implemeatation will 
be determined by- the- nature of the computer on wl^'ich they .are 
implemented and may not be. tj-ue of the "confputer" in our heads. 

In certain theoretical senses, investigations of abstract 
automata, such as Turing machine£ and abstract neural networks 
can tell us what kinds of functions various subparts of the brain 



might - perform' and what their limitations might, be . However,, we 

^ » - ♦ 

do , not yet have a- complete enough account of human intelligence 
in such -terms to derive practical Results Humans- are presumably 
heir- . to the same limitations that Turing machines' are known to 
have in that -they cannot possess algorithms- tp^^ aolve formally 
unsolvable 'problems', but .beyond that we cannot begin to derive 
such useful, predictions as haw much new • information^ per fninute 
can humans learn, how many facts can they store and remember, or 
any of the myriad practical questions that one would like to know 
to design effective, educational pedagogies. 

What AI can do is serve a role very mu^ like that which* 
.theoretical physics or chemistry serve' for thei^ , t\espectlve' 
fields. ft can suggest models that have fheoreticall 
characteri^ics that fit' the known data and predict unknown data. 
Xs ' such models^converge and begin to be supported by empirical 
study,, they can be put to a wide range of practical . uses , such as 
designing pedagogical strategies and "training material. However, 
the path between where- we are and the ability' to make such 
prediction^ will i^e\3uire a great deal of -Work. 

Computer models up t.o^ and. including the sentence level are 
now relatively well articulated and can be used quite well for 
^alogies with human processing. Above the sentence'level , 
however, current capabilities of computer systems are limited, 
and ^recognition of the intended interpre^tation of stories has==a 
number of characteristics that, make it fundamentally more 
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dif.ficdlt than . . -indiviaual sentence' parsing. ^ojt^ current 
attempts at. this level rely pn preselected scripts -that 
cons,train tiie possible interpretations of the sentences- that'they 
Mf. encounter to a microscopic f'ractioh of what, could otherwise 
occur. Sp.nce humaji. beings have encyclopedic i&mount's'^' knowledge 
that are drawn on and' used ^ ih "understanding 'what., they read , \ 
something much more than the current, sc'rip4:-based' theories will'' 
be required to deal , with ^neral human behavior.^ Ho#ver 
increasing interest, in this problem by linguists', computational 
linguists, philosophers, psychologists, and.^ researchers, in 
arti'ficial intelligence^ give promise ' for t^ development of 
increasi^ngly more adequate models, of the overall reading process. 
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